Add mistral-common library #1641

juliendenize · 2025-07-18T07:07:09Z

Hi !

We'd like to add vLLM as a library and track the downloads via two possible formats:

Transformers with the config.json
Mistral with the params.json

The code snippet suggested is through serving.

Edit:
Based on @Wauplin comment we changed the integration to instead add the mistral-common library

Wauplin · 2025-07-18T08:22:09Z

Hi @juliendenize , thanks for the PR. In the ecosystem we consider vLLM as a Local App rather than a library. If you go on https://huggingface.co/mistralai/Mistral-Small-24B-Base-2501?local-app=vllm for instance, you'll see how to serve the model using vLLM. If you are interested in counting the downloads of model repos based on mistral-common, I would suggest to register mistral-common instead. PR would look like this:

# (to add in alphabetical order) 
	"mistral-common": {
		prettyLabel: "mistral-common",
		repoName: "mistral-common",
		repoUrl: "https://github.com/mistralai/mistral-common",
		docsUrl: "https://mistralai.github.io/mistral-common/",
		countDownloads: `path:"config.json" OR path:"params.json"`,
	},

(I haven't added a code snippet as I'm unsure how to use mistral common)

You'll also have to add library_name: mistral-common in the model card metadata of your models.

Wauplin · 2025-07-18T08:24:31Z

Regarding the PR itself, we won't merge it in its current form. vLLM is a widely used app that is not specific to mistral-common (see https://huggingface.co/models?other=vllm) so we don't want the snippet to include mistral-specific instructions for all of them.

Wauplin · 2025-07-18T08:27:22Z

Btw, vllm snippets are defined here. It is possible to update the install command to include pip install -U mistral-common for models with mistral-common tag (e.g. if model.tags.includes("mistral-common") ...)

juliendenize · 2025-07-18T12:11:52Z

Hi @Wauplin thanks a lot for your detailed feedback and given me the context and the pointers ! much appreciated.

so we don't want the snippet to include mistral-specific instructions for all of them.

This was indeed also a concern for me when I pushed the original PR, and also why I put Mistral related instruction in comments. So, very cool that your proposed solution avoids that.

I updated the PR accordingly.

I just have a question regarding how tags works, is the library_name considered as a tag ? meaning would that be True:

model.tags.includes("mistral-common") // = True ?

Or should I update the PR to check the library_name ?

Wauplin

Hey @juliendenize , model.tags.includes("mistral-common") is the correct way!

Wauplin · 2025-07-22T16:10:18Z

packages/tasks/src/local-apps.ts

+			"# Make sure you have the latest version of mistral-common installed:",
+			"pip install --upgrade mistral-common"
+		].join("");
+		dockerCommand = `# Load and run the model:\ndocker exec -it my_vllm_container bash -c "vllm serve ${model.id} --tokenizer_mode mistral --config_format mistral --load_format mistral --tool-call-parser mistral --enable-auto-tool-choice"`;


out of curiosity, what happens if one runs docker exec -it my_vllm_container bash -c "vllm serve ${model.id}" instead of having all the mistral arguments? In general we avoid putting too many options in the snippets like this

It wouldn't know how to load the model correctly so failing :/

wouldn't this be something to fix on vllm side directly? feels weird if all vllm-compatible models can be correctly served with vllm server <model id> except the mistral-common ones that requires a very specific command

(not against adding it, just wondering why it's not a default)

Sorry my previous info was not complete, here is the behavior:

if transformers weights exist, it will load the model with transformers format => not our official way

if not, it will fail
So vLLM cannot know how to load the models without specifying the args.
And we can't only look if mistral-common is installed as it's a hard dependency of vLLM.

Wauplin · 2025-07-22T16:10:54Z

packages/tasks/src/model-libraries-snippets.ts

+
+export const mistral_common = (model: ModelData): string[] => [
+	`# We recommend to use vLLM to serve Mistral AI models.
+pip install vllm


Is mistral-common usable "as-is" or only under vLLM ?

mistral-common is a processing library and we always advise people to use vllm to serve our models (except for GGUFs). vLLM internally integrates mistral-common and use it when we correctly configure the serving.

Wauplin

Looks good, thanks for the indications. Before merging, can you make sure some models are listed under https://huggingface.co/models?other=mistral-common ? To do that you need to tag them as such in the model cards (library_name: mistral-common) . You can set tags: - vllm separately if you'd like but it shouldn't be as library_name otherwise the download count rule introduced in this PR will be ignored (since we only take the rule from the "main" library)

packages/tasks/src/local-apps.ts

Co-authored-by: Lucain <lucainp@gmail.com>

juliendenize · 2025-07-23T12:31:54Z

@Wauplin Should be good ! thanks for approving and suggestions

Wauplin · 2025-07-23T13:08:05Z

Hey @juliendenize thanks for the iterations. Last thing, can you run the linter to fix https://github.com/huggingface/huggingface.js/actions/runs/16469991767/job/46561455759?pr=1641 please ?

juliendenize · 2025-07-23T13:12:59Z

Hey @juliendenize thanks for the iterations. Last thing, can you run the linter to fix https://github.com/huggingface/huggingface.js/actions/runs/16469991767/job/46561455759?pr=1641 please ?

Yeah sorry about that @Wauplin just ran pnpm -r format as indicated in the contrib guidelines 🙏

Wauplin · 2025-07-23T13:16:46Z

Merged! Expect a few days before getting it live on the Hub :)

Add vLLM library

4e23a83

juliendenize requested review from SBrandeis, gary149, Wauplin, julien-c, pcuenca and ngxson as code owners July 18, 2025 07:07

Add mistral-common library

4cd3f27

juliendenize changed the title ~~Add vLLM library~~ Add mistral-common library Jul 18, 2025

Wauplin reviewed Jul 22, 2025

View reviewed changes

Wauplin approved these changes Jul 23, 2025

View reviewed changes

packages/tasks/src/local-apps.ts Outdated Show resolved Hide resolved

packages/tasks/src/local-apps.ts Outdated Show resolved Hide resolved

juliendenize and others added 2 commits July 23, 2025 13:57

Update packages/tasks/src/local-apps.ts

df8ee13

Co-authored-by: Lucain <lucainp@gmail.com>

Update packages/tasks/src/local-apps.ts

98ada82

Co-authored-by: Lucain <lucainp@gmail.com>

format

4297132

Wauplin merged commit c88fdc4 into huggingface:main Jul 23, 2025
3 of 4 checks passed

Add mistral-common library #1641

Add mistral-common library #1641

Conversation

juliendenize commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Wauplin commented Jul 18, 2025

Uh oh!

Wauplin commented Jul 18, 2025

Uh oh!

Wauplin commented Jul 18, 2025

Uh oh!

juliendenize commented Jul 18, 2025

Uh oh!

Wauplin left a comment

Choose a reason for hiding this comment

Uh oh!

Wauplin Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

juliendenize Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

Wauplin Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

juliendenize Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Wauplin Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

juliendenize Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

Wauplin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

juliendenize commented Jul 23, 2025

Uh oh!

Wauplin commented Jul 23, 2025

Uh oh!

juliendenize commented Jul 23, 2025

Uh oh!

Uh oh!

Wauplin commented Jul 23, 2025

Uh oh!

Uh oh!

juliendenize commented Jul 18, 2025 •

edited

Loading

juliendenize Jul 23, 2025 •

edited

Loading